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1 0 INTRODUCTION 


The satellite data acquisition and handling system currently implement- 
ed by NASA is now operating at capacity and as such provides severe limi- 
tation on data throughput NASA's mission model for the near-to-medium 
future indicates significant increases and/or changes in the data acquisition 
systems, data rates, and user requirements To anticipate this new era of 
space observation requirements, NASA is embarking on the NASA End-to-End Data 
System (NEEDS) program This program is an attempt to significantly increase 
the effectiveness and efficiency of the system that couples the user of space 
data with the sensors that acquire this data The NEEDS program will there- 
fore address the identification, development, and demonstration of data hand- 
ling and processing techniques and technologies which are required to accom- 
plish this 

More specifically, the NEEDS program goals present a requirement for on- 
board signal processing to achieve user-compatible, information-adaptive data 
acquisition One very specific area of interest, which this study addresses, 

IS the preprocessing required to register imaging sensor data which has been 
distorted by anomalies in subsatellite point position and/or attitude control 
This study brings attention to the concepts and considerations involved in 
using state-of-the-art positioning systems such as the Global Positioning Sys- 
tem (GPS) in concert with state-of-the-art attitude stabilization and/or de- 
termination systems to provide the required registration accuracy Aspects 
of the study include an examination of the accuracy to which a given image 
picture-element can be located and identified, the determination of those al- 
gorithms required to augment the registration procedure, and consideration of 
the technology impact on performing these procedures on-board the satellite 
The signal processing functions comprise a major constituent of the Information 
Adaptive System (IAS), a significant module of the NEEDS concept The IAS 
essentially consists of the spaceborne portion of NEEDS exclusive of telemetry, 
support and housekeeping systems A block diagram of the IAS is shown in Figure 
1-1 

The focus of volume one of this report was design for high-speed, high- 
resolution Landsat-D image preprocessing, namely, radiometric correction and 
geometric correction Many alternative approaches were considered, particular- 
ly in the area of resampling for geometric correction After the IAS hardware 
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Figure 1-1 INFORMATION ADAPTIVE SYSTEM BLOCK DIAGRAM 




IS implemented, the focus must shift to performance testing and evaluation, 
the subject of this volume 

The ideal procedure would be to test the processor on-board, while stor- 
ing raw detector outputs for subsequent high-accuracy software processing 
The cost of doing this, monitoring all functions, would, of course, be prohi- 
bitive This volume addresses the problems of simulation of actual data and 
performance evaluation 

The conclusion of this study is that the most significant problems in 
testing the IAS hardware are obtaining representative image and ancillary data 
and transferring and storing, at very high speed, a large quantity of image 
data 

It IS recommended that the former be handled by using a battery of model, 
aircraft and Landsat imagery together with ancillary data corresponding to ex- 
tremes of satellite pointing and ephemens errors 

The latter can be accomplished with modern high speed, high density digit- 
al tape recorders The recorders are generally quite expensive, and for limited 
evaluation may be replaced by high speed Random Access Memory (RAM) 

Use of commercial products or names of manufacturers in this report does 
not constitute official endorsement of such products or manufacturers, either 
expressed or implied, by the National Aeronautics and Space Administration 
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2 0 SYNTHETIC IMAGERY 


Such processing characteristics as speed and registration accuracy have 
requirements defined in the IAS specifications, but there is no optimal pro- 
cessing technique, and, thus, it is desirable to test and display the quali- 
ty of the processing beyond the mere verification that the specified parame- 
ters are within tolerance For example, it is useful to test and display 


edge accuracy 

overall interpolation accuracy (in spatial and spectral 
domains) 

classification accuracy 
registration accuracy 
interband registration accuracy 
noise rejection 
overf 1 ow/underf 1 ow 
processing speed 

Realistic imagery is, of course, best for testing, but it is probably not 
optimal for display, because of the limitations of the human visual system 
Indeed, subjective tests of image quality can lead to entirely different re- 
sults from objective tests of intensity, accuracy and resolution Thus, 
graphic displays of the results of objective tests are needed For example, 
the difference between a hardware-processed image and a very accurately soft- 
ware-processed image might be displayed as a difference image or color over- 
lay, much as with change detection processing The subjective judgement is 
thereby largely eliminated Moreover, this particular technique has the addi- 
tional advantage that it minimizes the effects of possible inconsistent dis- 
play characteristics 

It IS well-known that significant trade-offs are involved in registration 
processing, notably 

processing speed vs accuracy 
edge accuracy vs noise rejection 

classification accuracy vs overall interpolation accuracy 

The latter trade-off is implicit in the test results found in [1] One 
cubic convolution interpolator was found to be superior for classification only 
for small (1 or 2 pixel) areas and inferior for larger areas compared with 
simple linear interpolation To some extent, these trade-offs, too, can be dis- 
played 
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Realistic imagery should comprise the equivalent of several varied frames 
of 30m resolution earth images Each image should be available in several of 
the proper spectral bands and should be digitized to 8 bits of radiometric res- 
olution It should be geometrically corrected to some map projection so that 
it can be processed to a digitized Landsat-perspecti ve image Each frame should 
consist of at least 6133 pixels in each row and in each column The data should 
be available on IAS demonstration hardware compatible media as well as 9 track 
computer compatible tape 

Digitized image models are similarly restricted, except that ground dis- 
tance and resolution have no meaning for these images 

With these requirements in mind, the possible sources of digitized synthet- 
ic imagery will now be considered, together with the problems inherent in the 
use of each Any imagery for this purpose is synthetic, to some degree, since 
the hardware is to be flown on a new platform, with a new sensor and new optics 

One obvious candidate as a source of data is existing Landsat MSS (Multi- 
Spectral Scanner) imagery with its 79m x 57m pixel spacing and 79m ground reso- 
lution and its somewhat limited set of spectral bands Another possibility is 
Landsat RBV (Return Beam Vidicon) data with a resolution of 40m, and a single 
(visible) spectral band 

Various researchers have commented that the lower resolution imagery 
IS inadequate for a full-scale processing performance test and this seems to be 
a reasonable conclusion For example, registration accuracy depends upon inter- 
polation accuracy, which depends, in turn, upon the spectrum of the data being 
registered Interpolation can be regarded as a two-stage process the interpo- 
lation responses to each of the sample impulses are first combined to define an 
imperfect analog reconstruction of the original analog data and this reconstruc- 
tion IS sampled where interpolation estimates are needed The finite impluse 
of the interpolator is not bandlimited, and the reconstruction is not, in gener- 
al, a truncation of a bandlimited function There may, however, be a finite-di- 
mensional vector space of functions which are reconstructed exactly (except for 
quantization error) This occurs, for example, with the impluse responses corre- 
sponding to trigonometric polynomial interpolation and to linear interpolation, 
although this result is primarily of theoretical interest For most data, the 
interpolation is inexact If the original analog data were bandlimited, then 
the errors can be characterized as "resolution error" (in-band spectral error 
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of reconstruction) and "interpolation error" (out-of-band spectral error) [2] 

If the reconstruction is sampled at the Nyquist rate for the original data, 
there will, in general, be aliasing 

The preceding development has shown that interpolation is data-dependent 
One IS therefore forced to use high-resolution data when the registration 
accuracy associated with the interpolator is in question, for each spectral 
band, over the range of data to be encountered Once the interpolator has 
been chosen, it suffices to verify that it has been faithfully implemented 
The particular data dependence must then be accepted 

There will, however, be quite a variation in image spectra from deserts in 
one spectral band to urban centers in another spectral band One might antici- 
pate that this range would have a considerable overlap in the set of image spec- 
tra for lower resolution data One might be able to identify lower-resolution 
images which represent the extremes for registration accuracy low signal-to- 
quantization noise ratio (probably rural) and high spatial frequencies (prob- 
ably urban) The latter is favorable for registration accuracy, if an accurate 
interpolator is used A significant difficulty is the lack of 8 bit quantiza- 
tion in existing Landsat data, which results in a larger quantization noise 

To summarize, Landsat imagery could be used for testing (by treating a 
352 km swath of a mosaic as a 185 km swath of 30 m resolution imagery) for 4 
(5 for Landsat 3) spectral bands, but it has lower spatial and radiometric reso- 
lution and the image spectra may not be representative The registration and 
classification accuracies may differ from their true values However, these 
images are readily available and the full range of possible image spectra may 
provide an adequate test of registration and classification accuracy, especial- 
ly in view of the fact that differences would appear principally in the high 
spatial frequencies, where images typically have low energy 

The sampling rate of low resolution imagery is not high enough to permit 
the use of restoration techniques (it is not simply "blurred", it is blurred 
and undersampled) There would be some possibility of restoration to a 30 m 
resolution from several independent frames, but the exact form of the processing 
does not seem clear The simple approach would be to register the images with 
pixel-fraction offsets, combine the data into one high samplo-rate image, and 
then de-blur it Restoration from several images degraded by atmospheric blur 
has been attempted, i/ith some success 
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Another possible source of imagery is the Skylab S-192 Multi -spectral 
Scanner Experiment The ground resolution is that of Landsat (79 m) and 
imagery is very limited, but spectral bands very close to those to be tested 
were used Otherwise, the difficulties in utilization of this imagery are 
very similar to those encountered with Landsat imagery 

Some of the U S Geological Survey high-altitude aerial photographs could 
also be digitized for this application Visible and near-IR photographs are 
available The ground resolution is very good, but a large number of images 
must be mosaicked to obtain the 185 km size of a Landsat frame If the images 
can be digitized and mosaicked, they should be very good test images for their 
spectral band 

In addition to earth imagery, one might also use popular image processing 
test imagery, primarily to display properties of the processing These test 
images are frequently only 512 x 512 pixels and would be combined as a mosaic 
for processing An advantage of this test imagery is that errors in process- 
ing of familiar subjects are easily discerned The spectra of non-earth imagery 
will be somewhat different, of course, from that of realistic data 

Useful image models would include bars and checkerboard test patterns of 
1, 2, and 4 pixel pulse widths (periods of 2, 4 or 8 pixels, respectively), al- 
ternating between the minimum and maximum intensity values These will provide 
graphic visual displays of the processing and will test the overflow/underflow 
capabilities of the hardware (with the baseline interpolator, the interpolated 
intensity is sometimes outside the range of sample values used for interpolation, 
in contrast with simple linear interpolation) The 1 and 2 pixel square check- 
erboards are especially useful for displaying the degree of tradeoff between 
noise rejection and edge accuracy 

Another useful part of the test complement is a digitized image derived 
by sampling a bandlimited analog function which can be interpolated exactly 
from a finite set of samples, i e , a trigonometric polynomial 

A function like f(x) = [a. + a, Cos ]_ (2t7x) + a, Cos 1 (2ttx) + a, Cos 3 (2ttx)] 

° ' 8 4 '^8 

(in each dimension), which tends to represent the bandpass of the baseline inter- 
polator, would be useful It can always be recovered exactly from 8 regularly- 
spaced samples by trigonometric polynomial interpolation The normalized fre- 
quencies of the sinusoids are w = 0, 1, 1, _3, respectively 

8 4 8 
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By varying the magnitudes of the components k = 0, 1, 2, 3, and then 
measuring the registration accuracy, one can determine the effect of image 
spectrum upon registration accuracy (this analog model can be registered per- 
fectly, except for quantization error, but there will be some error with any 
4 point interpolator) The measurement of sub-pixel registration accuracy im- 
plies correlation of a hardware-processed sub-image with an ideally-processed 
sub-image, followed by some sort of curve fitting to determine the position of 
peak correlation 

For the reasons discussed above it seems reasonable to recommend the use 
of a battery of test imagery including a digitized mosaic of aircraft imagery 
in at least one spectral band and the image models described above, for dis- 
playing properties of the registration hardware processing 
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3 0 SYNTHETIC ANCILLARY DATA 


Sensor position and attitude, expressed in ECR coordinates, can be used 
to convert a map image to a Landsat-perspective image Any realistic syn- 
thetic imagery must be processed in this way, although synthetic model imagery 
or non-earth imagery will not be perspective-processed 

The geometry of the situation is elementary, the sensor position and atti- 
tude, together with internal sensor parameters, define an earth-sensing direc- 
tion for each individual detector of the array, and the ray in this direction 
intersects the surface of the reference ellipsoid at a unique position on the 
near side of the earth The intensity at this position can be estimated from 
the map image by accurate interpolation A good candidate for this accurate 
interpolation is 6 pixel by 6 pixel (non-penodic) cubic spline interpolation 
defined by a convolutional kernel h(d) 

for |dl<_l 
for l<|d|£2 
for 2<|d|j<3 
for |d|>3 

where d is the sample point to estimation point distance in pixels The re- 

A 

construction f of a sampled analog function f effected by this kernel h is 
f(x) h(x-n) 

n=- 00 


247|d|3-453|d|2-3|d + 209 
209 


h(d) = 


-114|d|3+612ld|^-1038|d| + 540 
209 


19|d|3-159|dP+434|d| -384 
209 


(of course, only a finite number of non-zero terms actually appear) This 
interpolator has roughly the same passband as the baseline interpolator, with 
less passband ripple 

One could ask if it might be better to use a large number of points in 
the interpolation In general, the answer is no, because spectral characteris- 
tics vary throughout an image, image intensities may vary smoothly in one area 
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and there may be a lot of detail in another (regarded as a stochastic process, 
an image is not "stationary") The image's overall spectrum would be accurate, 
at the expense of local accuracy An isolated edge, for example, would be 
smoothed by large-scale DFT-based interpolation 

A very practical advantage of this interpolator, not at all obvious from 
the definition of h(d), is that two of the six interpolation weights are al- 
ways simply- related to two of the others by the equation 

h(|d| +1)= - h(ldl) for l<|d|<2 
6 

This method has been tested on Landsat imagery and clearly provides good re- 
sults, although error analyses have not been carried out An eight point inter 
polator might also be considered for this application 

Whichever method is chosen, a digitized map image can be accurately dis- 
torted to a Landsat perspective, for processing by Landsat hardware The re- 
sult should be a map image of the same area (there will be some rotation and 
translation of the input map because of differing satellite orbits) 

The hardware operations may then be duplicated in software to provide a 
consistency test A test of absolute accuracy is defined by using accurate in- 
terpolation to transform the original map image directly to the hardware output 
coordinate system Then, on the average, the image degradation resulting pure- 
ly from the more accurate interpolator will be similar for the hardware-process 
ed image and the software-processed image Alternatively, the map image could 
be treated artificially as a perspective-distorted image, and fed directly to 
hardware and software processing (this completely eliminates the effect of the 
tendency of the accurate interpolator to smooth the input data while interpo- 
lating It) One could generate a corresponding artificial map image by accu- 
rately interpolating to "remove the perspective distortion" implied by sensor 
position and attitude 

The preceding discussion has indicated the type of data needed to regis- 
ter an image the sensor position, the sensor attitude, and the viewing di- 
rection of each detector relative to a sensor reference, all in ECR coordinates 
It should be remarked that these must be known for each swath of the frame 
Since this is a push-broom design, it may be assumed that these parameters vary 
slowly, and probably linearly, within a frame 

The direction of a detector's sensing of the earth can be adequately 
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modeled for the test as being in the direction of a single optical center In 
operation, it may be necessary to account for aberrations from this ideal 

It IS also important to note that these parameters would probably not be 
directly available Likely sources for these parameters are 

time and platform position, velocity — output of Kalman filter using 

GPS updates 

platform attitude, attitude rate -- output of Kalman filter using 

an advanced star-tracker and 
gyroscopic stellar-inertial 
reference for updates Accu- 
rate time used to put attitude 
vector into ECR coordinates 

sensor-platform alignment — temperature-dependence modeled 

as a function of time and 
calibrated occasionally using 
landmarks 


If errors in these parameters can be measured, then compensation is possi- 
ble The compensation may be physical (e g , thrust for attitude correction) 
or by means of processing (e g , discarding data from outside the desired earth 
area or, for sub-pixel errors, resampling interpolation) 

Errors in these parameters have different effects on the various types or 
registration 


relative registration — 


a postertoxn- absolute registration — 


a pmorz absolute registration -- 


a standard reference perspective, 
but no exact locations are 
specified 

a standard reference perspective, 
and at least some of the points in 
the image are identified with 
earth locations 

a standard reference perspective, 
and certain earth locations appear 
at pre-specified positions in the 
image 


The latter requires the highest degree of error measurement and correction, 
of course The standard reference "perspective" may be a true perspective (e g , 
that of a satellite at exactly 700 km), a well-known map projection (e g , Uni- 
versal Transverse Mercator) or a map projection especially selected for efficient 
registration processing or good scale accuracy throughout a satellite's sensing 
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swath For Landsat, a special projection is probably necessary Since a map 
projection will be used, the image coordinates are approximately proportional 
to true ground distance on the reference ellipsoid (for very precise registra- 
tion, it IS necessary to account for terrain relief, but this requires an ex- 
tremely precise earth model) 

Two images which are relatively registered can be brought into near coin- 
cidence by some unspecified translation and rotation (there will be a slight 
scale distortion in general) If the images are a poster^or^ absolutely regis- 
tered, then the necessary amount of rotation and translation can be easily cal- 
culated If they are a absolutely registered, they are automatically 

in coincidence 

There are two reasons for this digression into types of registration 
First, high-speed on-board image registration is probably a compromise level 
designed to satisfy a large number of users Second, a graceful degradation to 
a lower level of registration is desirable when some part of the system fails 
For these reasons, it is useful to explore the sensitivity of various levels of 
registration to errors in the basic parameters (An example of a partial fail- 
ure's reduction of registration level is the 2 1 km line start error in Land- 
sat 3 [3] ) The shift reduces the likelihood of ground control point location, 
and registration degrades toward relative registration 

It appears there are two possible sources of synthetic data needed for 
registration of image data real data from a similarly-stabilized satellite 
platform or simulated data based upon the design tolerances The latter is to 
be preferred for several reasons Real data may not be representative of opera- 
tional data for the proposed system Moreover, the proposed system will use 
GPS and an advanced star- tracker to achieve measurement accuracies not now attain- 
able Finally, for test purposes, one is interested in stressing cases, not a 
small sample of "representative" data One examines the sensitivity to a large, 
accurately-measured platform position or attitude error, for example 
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4 0 IAS INTERFACES 

Image data and ephemens and attitude data must be fed to the image regis- 
tration hardware for the purposes of the test No ordinary 9 track tape drive 
IS fast enough to supply the image data to the hardware, so the image data must 
be put onto another medium If one assumes that there are 16 lines of 6133 de- 
tectors in the multiple linear array, for each of 7 spectral bands, and that 
each intensity is coded with 8 bits of resolution, then the bit rate is 

16 X 6133 X 7 X 8 bits per second 

T 

where T is the duration of one transfer cycle For example, if T = 0 071 se- 
conds, then the required bit rate is 76 8 megabits per second This is roughly 
the lower bound for the bit rate, it will be higher if data transfer must be 
stopped for some portion of the total cycle time 

The straightforward, but impractical, approach is to transfer the data to 
a semiconductor memory having a high data rate, in analogy with the operation 
of reading a portion of a magnetic tape into the core memory of a computer 
The problem is the tremendous storage requirement, approximately 266 megabytes, 
that of a large disk pack Double buffering could reduce the storage require- 
ment by a factor of 2 That is, if the memory initially contained 133 mega- 
bytes, It could be rewritten at one part while it was being read at another, 
until all 266 megabytes were transferred 

Multiple access read/write capability could reduce the individual data rate 
requirement, multiple access is natural, since there are 7 spectral bands and 
approximately 16 lines of detectors for each band 

It would be feasible to use this method for a small fraction of an image 
frame, especially if the segments could be combined later to simulate full frames 
of hardware-processed output It would seem that two sensing cycles of data (two 
"scans") would be a minimum requirement for an acceptable test, since proper buff- 
ering and processing between scans are significant functions to be tested 

Probably the most cost-effective design is one based on a very-high-speed 
tape drive, although the required data rates are near the technology limit for 
these devices (see Appendix) 

Another possibility is a very high speed disk drive The flexibility of a 
random-access device may, however, be wasted in this application, since a frame 
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of image data would probably always be accessed in the same sequence 

A less acceptable alternative is to use a high-resolution film recording 
system like that presently used at NASA-Goddard A mirror-scanned laser gen- 
erates a high-resolution latent film, which then undergoes photographic pro- 
cessing (see Section 41) An inverse system, wherein a scanned laser sends 
light through a selected portion of a positive transparency to be sensed by a 
calibrated detector, might be used to read intensities from such a film The 
disadvantages are the time requirement and the analog nature of the storage 
The present bit rates are adequate for each spectral band, but there are only 
64 radiometric resolution levels 

4 1 NASA High Resolution Film Recording Subsystem 

The high-resolution film recording subsystem shown in Figure 4-1 uses an 
argon ion laser light source (argon lasers operate at 488 0 and at 514 4 nm) 
and a rotating mirror to expose film in a line-scanned format The black and 
white film image is about 190 mm wide The exposure of the film by the modu- 
lated laser is synchronized with the continuous movement of the film The steps 
of the process are 

•digital adjustment for processing non-linearity 

•transfer to high-speed buffer 

•digital-to-analog conversion 

•synchronized exposure of the film to modulated 
laser light through precision spot-forming optics 

As presently implemented, this is a digital-to-photographic transition only 
For more details, see Table 4 1 and [4] 
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Table 4 1 High-resolution Film Recorder Performance Characteristics [4] 


Function/Operation 

Performance Objectives 

Film Writing Area 

Frame Width 206 2 mm (active scan width) 
202 2 mm (Image width) 


Frame Length — Variable, depends on input 
array size HRFR can also run in a continuous 
strip mode 

Spot Size 

10 to 200mm 

Input Data Rate 

0 2 to 20 Mbits/sec (continuously variable) 

Line Rate 

60 to 350 lines/sec 

MTF 

>50% for 20,000 pixels per line 

Line Spacing 
(Transport Jitter) 

< l/im rms 

Scan Jitter 
Start of Scan 

< 5mm rms 

Dynamic Range 

100 1 (modulator output) better than 2 00 on 
Kodak 2460 or 3414 film 

Grayscale 

>64 distinguishable steps 
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5 0 REFERENCE SOFTWARE PROCESSING 


The alternatives for the testing of hardware processing are subsequent 
"de-processing" or parallel processing Since registration interpolation is 
inherently irreversible, the latter alternative is recommended The irrever- 
sibility of interpolation is seen even with simple linear interpolation in a 
table For example, the table 


X 

1 

2 

3 

y 

0 

1 

0 


can be interpolated to obtain y=05atx=l 5 and at x = 2 5 


X 

1 5 

2 5 

y 

0 5 

0 5 


If this table is interpolated, it is found that the estimate at x = 2 provided 
by interpolation is y = 0 5, not the original value Smoother functions (that 
IS, functions with less high spatial frequency content) can be "de-interpolated" 
more accurately, but there is generally some error (of course, a linear function 
will be interpolated and de-interpolated exactly by linear interpolation) 

Now, having decided to implement parallel processing, one must consider 
data sources and exact procedures The problems of simulating image data and 
ancillary data have already been discussed The calculation of the displacement 
of a sample point from its intended location is done using the ancillary data 
and certain functional approximations There are unavoidable truncation and 
round-off errors, and the extent of these errors will depend to some extent upon 
the magnitudes of the position and attitude errors to be compensated for 

After the sample displacement has been calculated, an estimate of sample 
intensity is determined by resampling interpolation, whose accuracy in represent- 
ing the continuous data is dependent upon the nature of the continuous data The 
latter depends upon optics and detector resolution, as well as the intensity dis- 
tribution of the imaged area (it is assumed that the detector readings have been 
radiometncal ly-corrected before interpolation is carried out) The nature of 
the data also depends upon the spectral band being sensed 

Since data which represent high resolution Landsat data very accurately are 
difficult or impossible to obtain, a sub-optimal approach will be described The 
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image data are simulated by several frames of current Landsat MSS data, repre- 
senting the range of image spectra, from very slowly varying to those with 
significant hi gh-spatial -frequency content A small (say, 32 x 32 pixel) sub- 
image of each frame is selected and a two-dimensional trigonometric polynomial 
IS fit to the subimage, using the discrete Fourier transform The trigonometric 
polynomial may then be used to test registration accuracy If the registration 
accuracy is relatively insensitive to the choice of the Landsat image, one 
would anticipate that the interpolator would yield similar registration accura- 
cies with true high resolution images, if the accuracy varies, a rough estimate 
of registration accuracy is obtained 

The procedure for measuring registration accuracies is as follows Choose 
a subimage of each frame of geometrically-corrected data and fit a trigonometric 
polynomial to it Sample the trigonometric polynomial at various offsets After 
converting the geometrically-corrected data to simulated raw data according to 
the simulated position and attitude, and hardware-processing it, use correlation 
and curve-fitting to estimate the position of the trigonometric polynomial sub- 
image (offset by a fraction of a pixel) in the resulting processed image 

One comment is in order about the use of realistic imagery, relative to the 
use of the image model suggested in Section 20 If it is to be treated as a 
sampled two-dimensional trigonometric polynomial, some consideration should be 
given to the OFT bias toward subharmonics of the sampling frequency (harmonics 
of the truncation frequency) resulting from exact periodic truncation 

The use of a spectral estimation digital window, such as a Blackman or Kai- 
ser window, would offset this bias, putting the energy of frequencies between 
subharmonics into close subharmonic components Without windowing, the energy 
in non-subharmonics tends to be spread throughout the spectrum [5] Subharmonic 
energy is confined to narrow frequency intervals, which are widened slightly by 
windowing 

Thus, It IS recommended that, when a trigonometric polynomial is to repre- 
sent realistic image data, the data should be windowed The reference digital 
data are then the windowed realistic data and the reference analog data are repre- 
sented by the trigonometric polynomial fit to this data using the DFT 
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6 0 PERFORMANCE MEASURES AND EVALUATION 

There are at least three levels of testing functional testing, consis- 
tency testing and absolute error testing The first relates to the nominal 
operation of a given function under simulated operating conditions, that is, 
outages or gross errors, especially at the very high speeds involved 

The second is concerned with interpolation, registration, and classifica- 
tion accuracy relative to a software- implemented baseline reference 

The third involves the absolute errors incurred while sampling and process- 
ing an analog or near-analog signal (e g , a very high resolution image) This 
third level of testing requires the most realistic synthetic imagery, although 
there are special cases, such as overflow and edge effects, which must be con- 
sidered in hardware processing even at the second level of testing if it is to 
be done properly 

The third level of testing could be accomplished by very careful, detailed 
software simulation of the IAS image processor's operation, given that the IAS 
hardware processing must be consistent with the baseline reference The simu- 
lation would, or course, take into account the exact timing of processing opera- 
tions It could be used, for example, to test changes in the radiometric correc- 
tion or interpolation processes, without hardware implementation However, each 
facet of the intricate processing logic would have to be tested for consistency 
with hardware operation Such things as buffering to permit interpolation across 
adjacent multiple linear sample sets would have to be checked very carefully A 
more direct approach is to test the hardware processing at two levels consis- 
tency with the software baseline, and absolute accuracy relative to an analog sig- 
nal Neither of these requires a detailed software simulation, but, on the other 
hand, only minor variations in the hardware processing, such as in the interpola- 
tion and radiometric correction parameters, can be made 

The functional testing is quite straightforward, since the functional 
components of the IAS are well defined A function of the adaptive system 
controller, namely selective processing bypass, will be useful for various 
comparisons 

For the second level of testing, a baseline reference for processing will 
have been established The exact form of the radiometric correction algorithm 
will be known The exact interpolation algorithm and map projection will be 
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explicitly defined The radiometric correction and geometric correction can 
be performed independently in software, with the same degree of precision, to 
verify that the expressed computations are, in fact, being carried out, even in 
exceptional cases of large intensities and scan extremes If there are discrepan- 
cies, they may occur in the "distortion calculation" or in the resampling inter- 
polation operation These checks are largely content-independent, except that the 
dynamic range of the processor should be tested Since absolute equality is to 
be expected, all errors should be considered significant 

Assuming that the hardware passes the first two levels of testing, one can 
move on to the testing of absolute accuracy, from several different viewpoints 
The most important of these are interpolation accuracy, registration accuracy and 
classification accuracy It is at this point that the realism of the simulated 
imagery becomes very important, since these components of the processing are some- 
what data-dependent The tests make sense for test images generated, for example, 
as linear combinations of two-dimensional sinusoids, but the results would not 
provide a thorough test 

It may be useful to process checkerboard imagery with blocks of 1 , 2, and 
4 pixel sizes, respectively, for the display of square wave interpolation charac- 
teristics This is not, however, intended to represent real data, it tests under- 
flow and overflow and gives a general indication of the bandpass characteristics 
of the interpolator Note that interpolators using only two sample points, for 
example, linear interpolation, will do well on this test (the "data" in this case 
are far from bandlimited) 

A more useful type of synthetic imagery uses sinusoidal variation in each 
dimension It is more meaningful to consider interpolation accuracy here, be- 
cause the "data" can be bandlimited It should be noted that there are two types 
of interpolation accuracy absolute accuracy and accuracy in the mean As one 
example, consider trigonometric polynomial interpolation at a normalized frequency 
0 ) = 0 5 If the sinusoid is sampled at its zeros, all estimates will be zero If 
the sinusoid is sampled at its extrema, it will be interpolated correctly On the 
average, there is a 50% error Thus, the spectral response H((u) = 05ato) = 05 
At oj = 0 25, however, the interpolation is always exact and H(0 25) = 1 0 Another 
example is linear interpolation Since curvature is never estimated, no non-triv- 
lal sinusoid is ever reconstructed exactly, and lH(o))| <1 0 for each u =f= 0 A 
piecewise-linear reconstruction is far from bandlimited, and |H(aj)| is substantial 
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for large u, and is zero only for non-zero integer values 

The preceding demonstrates that Fourier analysis is not enough, since it 
does not distinguish between the two types of errors, even when the data are 
samples of pure sinusoids This suggests that interpolation accuracy should 
be checked directly Sinusoidal image models can be interpolated exactly and 
the hardware interpolation can be compared with the ideal Mean, mean-square 
and maximum disparities would be measured, and the frequencies of the sinusoids 
may be varied to examine spectral dependence 

Similarly, a linear combination of sinusoids may be used for a preliminary 
test of registration accuracy The ultimate test of registration accuracy must 
utilize realistic imagery The simply-modeled imagery does provide an opportun- 
ity to explore the spectral dependence of registration error 

Whatever imagery are used, registration to the nearest pixel can be accom- 
plished by correlation or by the minimum absolute difference method [6] To 
achieve sub-pixel registration, this step must be followed by a curve-fitting 
step to estimate the exact maximum correlation point or exact minimum absolute 
difference point from the pixel-space samples of these quantities The match- 
ing area could be a subimage of roughly 32 x 32 pixels 

Thus, the procedure is to process the test image exactly and by the hard- 
ware processor and register a hardware-processed subimage with the exactly- 
processed subimage which corresponds to it The estimated displacement is the 
registration error 

The difficulty with actual image data is that it consists of samples of a 
continuous data set which is no longer available, hence, can not be interpola- 
ted exactly If the data set can be assumed to be essentially bandlimited, then 
a DFT can be used to approximate ideal bandlimited interpolation If this is 
not to be assumed, then it is suggested that a 6-point cubic spline interpola- 
tor be used for processing the reference, since it has the same bandpass as 
the baseline 4-point periodic cubic spline interpolator, with much greater accu- 
racy in the bandpass region Most of the image energy is in this bandpass region, 
and the image will be interpolated quite accurately 

If several spectral band images are registered simultaneously, it is to be 
expected that the estimated location of a landmark in each spectral band will 
be slightly different, because of interpolation error This is especially sig- 
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mficant for the two spectral bands used m the FILE* algorithm for elementary 
classification 

Classification will not be considered in detail, except to note that the 
simpler interpolators, i e , nearest neighbor and linear, seem to work quite 
adequately for classification and, in fact, may be preferable to those which 
are more complex, in this respect 


* FILE (Feature Identification and Location Experiment) algorithms use spec- 
tral radiance ratio detection techniques to classify data into groups such as 
bare land, water, vegetation, or clouds and snow/ice A subsequent algorithm 
discriminates between clouds and snow/ice [7] 
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7 0 DISPLAY REQUIREMENTS 

The TSDE CRT should display processing status and the results of objective 
tests of processing accuracy, such as error histograms (relative to software- 
processed input data) Provisions should also be available for input or output 
display 

The display of input and output is essentially equivalent A portion of a 
frame (say, 1024 x 1024 pixels) could be displayed at one time for up to three 
of the Landsat spectral bands (false color composite) A two-color display of 
the spectral bands of the FILE classification algorithm would be especially in- 
teresting In general, these displays would be useful in the identification of 
basic imagery type and gross accuracy checks, rather than in performance compari 
sons 

The display of processing status might include such items as identification 
of input data, nature of the processing to be done, analysis to be performed on 
the resulting output and the exact stage of the processing which is being per- 
formed at the time The outputs to be displayed should be enumerated Gross 
error conditions or outages should be flagged 

Some of the performance measures to be displayed are error histograms for 
each spectral band, differential images for individual spectral bands (hardware- 
processed versus software-processed), the registration correlation function, and 
registration error The results of the FILE classification algorithm could also 
be displayed 


23 



APPENDIX A 

HIGH DENSITY DIGITAL RECORDING 
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A 1 Background- 

First, consider the problem of recording just one track of digital data 
on a tape If speed were not required, the magnetically-polarized "head", 
across which the tape moves, would simply convert the electrical pulses to 
sharply-defined regions polarized along the length of the tape Again, if 
speed were not required, the polarity of the tape region's magnetic flux could 
be measured as the tape moved very slowly past the read head In fact, for 
practical reasons, it is the rate of change of flux which must be measured, so 
that all transitions should be as sharp as possible for ease of identification 
Because of the record-playback bandwidth limitations, the transitions may not 
be sharp With pulse broadening, a long sequence which is heavily weighted 
toward I's may cause a 0 to be misread, because a transition is not sharp 
enough to be read correctly by the reproduce head Encoding schemes are used 
to ensure that the number of pulses of the two polarities remain approximately 
equal, so that the expected interference with a given pulse of either polarity 
IS small The data spectrum should thus have low energy near zero frequency 
("DC"), and, of course, it should have little energy above the bandpass of the 
record-playback system These conditions are associated with transition spac- 
ings which are somewhat uniform 

A 2 Encoding Schemes - 

There are many encoding schemes for the transmission and storage of digit- 
al data When noise, interference and distortion are present in the medium. 

It is necessary to maximize the difference between the finite number of states 
used to encode the data In the case of electrical or magnetic media, the 
code should use pulses of opposite polarity The simplest code of this type 
IS called Non-Return-to-Zero-Level (NRZ-L), which represents I's with one po- 
larity and O's with the opposite polarity The NRZ-Mark (NRZ-M) and NRZ-Space 
(NRZ-S) codes represent I's and O's by transitions of polarity or lack of tran- 
sitions Any of these codes can result in long strings weighted heavily with 
one polarity 

The premise of Randomized NRZ is that a random data set would not contain 
long strings which are weighted in favor of one polarity Nor would it contain 
any information The data may be scrambled, or randomized, into a pseudo-random 
sequence by a shift register and exclusive OR gate and the unscrambling requires 
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only a synchronized scrambler, acting as an unscrambler For many purposes, 
pseudo- randomness, as measured by any of several tests, is an adequate re- 
placement for "randomness" There are still data which encode to biased bit 
streams, but they are much less likely to occur than strings of O's or I's 
A great advantage of this code is that no (serial) overhead data storage is 
required However, the scrambler and unscrambler must start at exactly the 
same point, and any errors which do occur will persist for an entire feedback 
shift register cycle This method is the hardware solution to the encoding 
problem 

The Ampex Miller^ code is a variation of the Miller code, which uses a mid- 
cell transition to encode a 1 and a transition at the leading edge of the cell 
for a 0, except that there is no transition for a 0 following a 1 In the Miller^ 
code, there may not be a transition for the last of the I's in a subsequence of 
I's, if the subsequence has even length Apparently there is an "infinite mem- 
ory" to keep track of accumulated charge (another expression for bias), and 
transitions are suppressed or not, depending upon the current value of charge 
Thus, the charge can be maintained within the limits ±3, so that there is no 
long-range DC buildup Only a finite memory, a four-stage shift register, is 
needed for decoding 

The basic clock frequency with the Miller^ code is twice as high as with 
NRZ, but transitions are always spaced by at least two clock cycles Synchroni- 
zation to 1/2 bit period is required, so that mid-bit and edge-bit transitions 
can be distinguished 

The original Miller code has less interference from adjacent data, since 
run lengths are limited to two bit periods The Miller^ code has transition 
spacings of 1 , 1 1/2, 2, 2 1/2, or 3 bit cells 

Enhanced NRZ is a simple scrambling technique involving inversion of cer- 
tain bits in each 7 bit group Otherwise, it is NRZ-L encoding, except that a 
parity bit is added to each group of 7 bits The sensitivity is not to long 

strings of O's or I's but to another pattern The parity bits represent 12 5% 
serial overhead 

\ 

A 3 Additional Considerations - 

Of course, a higher serial bit rate can be achieved by using several 
tracks on the same tape, with a senal-to-parallel conversion for recording and 
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a parallel -to-senal conversion for reproduction (see Figure A-2) 



Figure A-2 Block Diagram of a Multi-Track High-Density 
Digital Recording System 

The processes are called skewing and de-skewing and a difficult synchronization 
problem is inherent in these processes, especially when the data are packed at 
densities of 33 3 kilobits per inch The overhead data may be stored in the 
same tracks as the true data, or in separate tracks There may be up to 40 
tracks of data on a single standard 1" tape 

Another parameter of interest is the total record or playback capability 
with a single reel of tape, since image processing applications usually require 
a massive collection of data to be recorded at one time An expression for tape 
length 1 as a function of reel diameter d and tape thickness t is 

1 = -f ) inches 

This expression assumes a 3" hub and that t«d For one mil total thickness, 
the expression simplifies to 

1 = 785 4 (d2-9) inches 

At 120 inches per second, a 16" reel may be used for about 1600 seconds, or a- 
bout 27 minutes If there are 28 tracks, there are about 6460 megabits of data 
on the tape, and the overall bit rate is about 112 megabits per second Over- 
head uses as little as 3-8% of this capacity 

A 4 Comparison of Systems - 

In Table A-1, each manufacturer is represented in two rows The upper row 
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contains maximum values of parameters for systems which have been developed 
These do not necessarily constitute a consistent set The bottom row lists 
the parameters of the system which would be applicable to the Landsat-D image 
registration hardware test 

The number of data tracks is the number of tracks which contain user data, 
even if overhead data are included on the tracks The data rate is the serial 
data rate for user data, except as indicated The tape capacity is based upon 
the maximum reel diameter specified The reproduce time is the tape reel ca- 
pacity divided by the tape speed 

As Table A-1 illustrates, the basic performance of the systems of the three 
manufacturers represented is comparable Major advances in speed and bit error 
rate should not be expected with this technology, although new recording tech- 
niques, such as those now being used to store analog video data, are under 
development 

The type of recording discussed here is direct, saturation recording At 
lower data rates, it is possible to represent bits by sinusoidally-magnetized 
regions of polarization The rate of change of a pure sinusoidal function is 
just a shifted version of the sinusoid The bandwidth of the record-playback 
system does not permit this sort of recording (FM recording) at high data rates 
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